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Commissioner for Patents 
P.O. Box 1450 
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Sir: 

In response to the Notification of Non-Compliant Appeal Brief dated August 29, 2006, 
Appellants submit this Supplemental Appeal Brief: 
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I. REAL PARTY IN INTEREST 

Based on the information supplied by the Appellant, and to the best of Appellant's legal 
representative's knowledge, the real party in the interest is the assignee, SAMSUNG 
ELECTRONICS CO., LTD and the Regents of the University of California. 



2 



SUPPLEMENTAL APPEAL BRIEF UNDER 37 C.F.R. § 41.37 
U.S. APP. NO. 09/916,210 



Q61834 



II. RELATED APPEALS AND INTERFERENCES 

To the best knowledge and belief of Appellant, the Assignee and the undersigned 

attorney, there are no other appeals or interferences before the Board of Appeals and 
Interferences ("the Board") that will directly affect or be affected by the Board's decision in the 
present Appeal. 
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III. STATUS OF CLAIMS 

Claims 1-8 are cancelled. 

Claims 9-11 are all the claims pending in the application. Claims 9 and 10 have been 
finally rejected, and are the subject of this appeal. Claim 11 would be allowable if rewritten in 
independent form, and Appellant submitted with the Appeal Brief filed on August 14, 2006, an 
Amendment rewriting claim 1 1 in independent form. After amendment, claim 1 1 is as follows: 

1 1 . (previously presented): An object activity recognition method comprising the steps 

of: 

(a) obtaining feature vectors by motion estimation for video frames; 

(b) determining a state, to which each frame belongs, using the obtained feature vectors; 

and 

(c) determining an activity model, which maximizes the probability between activity 
models and a video frame provided from a given activity model dictionary using a transition 
matrix for the determined state, as the recognized activity, wherein the step (c) comprises a step 
of finding an activity model, which maximizes probability P(0|A/) from the given activity model 
dictionary {k\, ^2, ^e}, when T is a positive integer indicating the number of frames forming 
the video sequence, Zi, Z 2 , Z T are feature vectors of first frame, second frame, T-th frame, 
respectively, and if video frame 0={Z U Z 2 , Z T } is given and E is the number of state models, 
and wherein the transition matrix is obtained by using an expectation-maximization (EM) 
algorithm based on the observation symbol probability {bj(.)} corresponding to scene j in the 
training process. 
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IV. STATUS OF AMENDMENTS 

No amendments have been made subsequent to the Final Office Action dated October 12, 
2005. However, Appellant submits with this Appeal Brief, an Amendment which rewrites 
allowable claim 1 1 in independent form. 
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V. SUMMARY OF THE CLAIMED SUBJECT MATTER 

An aspect of the invention provides an object activity recognition method. Referring to 

Fig. 3, an aspect of the invention obtains feature vectors by motion estimation for video frames 
(Fig. 3, S304), determines a state to which each frame belongs using the obtained feature vectors 
(Fig. 3, S306), and determines an activity model, which maximizes the probability between 
activity models and a video frame provided from a given activity model dictionary using a 
transition matrix for the determined state, as the recognized activity (Fig. 3, S308). 
Claim 9 recites: 

An object activity recognition method 
(see, e.g., page 1, lines 2-6) comprising the 
steps of : 

(a) obtaining feature vectors by motion 
estimation for video frames (see, e.g., page 
14, lines 7-10; Fig. 3, element 304); 

(b) determining a state, to which each 
frame belongs, using the obtained feature 
vectors (see, e.g., page 14, lines 11-12; Fig 
3, element 306); and 

(c) determining an activity model, which 
maximizes the probability between activity 
models and a video frame provided from a given 
activity model dictionary using a transition 



SUPPLEMENTAL APPEAL BRIEF UNDER 37 C.F.R. § 41.37 Q61834 
U.S. APP. NO. 09/916,210 

matrix for the determined state, as the 
recognized activity (see, e.g., page 14, lines 
12-19; Fig. 3, element 308). 
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VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

Claims 9 and 10 are rejected under 35 U.S.C. § 103 as being unpatentable over Martens 

et al (U.S. Patent No. 6,157,677) in view of Hu (U.S. Patent No. 5,748,247). 
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VII. ARGUMENT 

A. Rejection of claims 9 and 10 under 35 U.S.C. § 103 as being unpatentable over 

Martens et al (U.S. Patent No, 6 J 57,677) in view of Hu (U.S. Patent No. 5.748,247). 

Appellant respectfully submits that claim 9 is patentable over the applied references at 
least because the references commonly fail to teach or suggest "determining an activity model, 
which maximizes the probability between activity models and a video frame provided from a 
given activity model dictionary using a transition matrix for the determined state, as the 
recognized activity", and at least for this reason fail to render obvious the invention defined by 
claim 9. 

In the Office Actions dated April 20, 2005 and October 12, 2005, and the Advisory 
Action dated January 31, 2006, the Examiner refers to portions of Martens et al as disclosing 
each of the claimed features, except that he admits that Martens et al does not specifically 
disclose determining an activity model, which maximizes the probability between activity 
models and the video frame. The Examiner, however, states that Hu teaches refinement of 
motion vectors, and that an activity model has been used to maximize the probability that the 
derived motion vector field represents the truth as a commotion (recognized activity). The 
Examiner cites column 2, lines 33-38 of Hu as teaching this feature. (October 12, 2005 Office 
Action, 7 th paragraph of page 2; see, also the Advisory Action of January 31, 2006.) 

Appellant respectfully submits that the applied references, taken either alone or in 
combination, fail to teach or suggest the step (c) of claim 9, and at least for this reason fail to 
render obvious the invention defined by claim 9. 
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In more detail, Appellant submits that Martens et al does not teach or suggest 
maximizing, or even enhancing, the probability between activity models and a video frame 
provided from a given activity model dictionary . 

The Examiner cites col. 3, lines 10-19, col. 4, lines 39-45, col. 11, lines 41-48, col. 13, 
lines 66-67, and col. 14, lines 1-9 of Martens as teaching using a transition matrix from the 
determined state as the recognized activity from an activity model dictionary (5 paragraph of 
page 2 of the Office Action dated October 12, 2005). Appellant respectfully submits that the 
portions of Martens cited by the Examiner, however, merely teach using a model to enhance 
motion estimation. There is no teaching or suggestion of determining an activity model which 
maximizes the probabilit y between activity models and a video frame p rovided from a given 
video model dictionary using a transition matrix for the determined state, as the recognized 
activity . Also, Martens does not teach or suggest determining an activity model as a recognized 
activity. 

The Examiner refers to the Martens et al Abstract, column 13, lines 59-67 and column 14, 
lines 1-12, as teaching determining an activity model which enhances the probability between 
activity models and video frame using a transition matrix for the determined state, as a 
recognized activity. These portions of Martens et al, however, do not teach or suggest anything 
with respect to the claimed feature of the given activity model dictionary. Martens et al does not 
teach or suggest maximizing, or even enhancing, the probability between activity models and a 
video frame provided from a given activity model dictionary . The portions of Martens et al cited 
by the Examiner relate to correcting for systematic intensity changes that would otherwise 
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impede the motion estimation so that the probability of erroneously modeling motion effects in 
the intensity domain is minimized. There is no teaching or suggestion of maximizing or 
enhancing the probability between activity models and a video frame provided from a given 
activity model dictionary using a transition matrix for the determined state. 

Furthermore, even if it were to be conceded, for the sake of argument, that Martens does 
teach what the Examiner alleges Martens teaches, which Appellant believes it does not, the 
combination proposed by the Examiner would still not render the claimed invention obvious. In 
more detail, the Examiner states that Martens teaches that classification probability can enhance 
motion estimation and modeling, and that Hu teaches refinement of motion vectors and that an 
activity (Gibbs/Markov) model has been used to maximize the probability that the derived 
motion vector field represents the true physical motion (recognized activity) (Office Action, page 
2, paragraphs 6-7). But Hu merely teaches using a model to maximize the probability that the 
derived motion vector field represents the true physical motion. This teaches nothing about 
determining an activity model as a recognized activity; it merely teaches using one model to 
maximize the probability of obtaining an accurate motion vector field. Therefore, Appellant 
respectfully disagrees with the Examiner's statement that Hu teaches "maximizing the 
probability between activity models and the video frame" (October 12, 2005 Office Action, page 
2, second from last line). 

For at least the above reasons and the reasons presented in the previously filed 
Responses, Appellant submits that claim 9 and its dependent claim 10 are not rendered obvious 
by the applied references. Appellant therefore requests that the rejection of claims 9 and 10 
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under 35 U.S.C. § 103 as being unpatentable over Martens et al (U.S. Patent No. 6,157,677) in 
view of Hu (U.S. Patent No. 5,748,247) be withdrawn. 

Unless a check is submitted herewith for the fee required under 37 C.F.R. §41. 37(a) and 
1.17(c), please charge said fee to Deposit Account No. 19-4880. 

The USPTO is directed and authorized to charge all required fees, except for the Issue 
Fee and the Publication Fee, to Deposit Account No. 19-4880. Please also credit any 
overpayments to said Deposit Account. 

Respectfully submitted, 




SUGHRUE MION, PLLC Peter A. McKenna 

Telephone: (202) 293-7060 Registration No. 38,55 1 

Facsimile: (202) 293-7860 

WASHINGTON OFFICE 

23373 

CUSTOMER NUMBER 

Date: September 29, 2006 
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CLAIMS APPENDIX 

CLAIMS ON APPEAL: 

Claims 1-8 (canceled). 

9. An object activity recognition method comprising the steps of: 

(a) obtaining feature vectors by motion estimation for video frames; 

(b) determining a state, to which each frame belongs, using the obtained feature vectors; 

and 

(c) determining an activity model, which maximizes the probability between activity 
models and a video frame provided from a given activity model dictionary using a transition 
matrix for the determined state, as the recognized activity. 

10. The object activity recognition method of claim 9, wherein the step (c) comprises 
a step of finding an activity model, which maximizes probability P(0|X) from the given activity 
model dictionary {X\, A, E }, when T is a positive integer indicating the number of frames 
forming the video sequence, Zi, Z2, Zt are feature vectors of first frame, second frame, T- 
th frame, respectively, and if video frame 0={Zi, Z 2 , Z T } is given and E is the number of 
state models. 
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EVIDENCE APPENDIX: 

NONE 
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RELATED PROCEEDINGS APPENDIX 



NONE 
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